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Abstract. This article presents several results establishing connections be- 
tween Markov chains and dynamical systems, from the point of view of open 
systems in physics. We show how all Markov chains can be understood as 
the information on one component that we get from a dynamical system on a 
product system, when losing information on the other component. We show 
that passing from the deterministic dynamics to the random one is character- 
ized by the loss of algebra morphism property; it is also characterized by the 
loss of reversibility. In the continuous time framework, we show that the solu- 
tions of stochastic differential equations are actually deterministic dynamical 
systems on a particular product space. When losing the information on one 
component, wc recover the usual associated Markov semigroup. 



1. Introduction 

This article aims at exploring the theory of Markov chains and Markov processes 
from a particular point of view. This point of view is very physical and commonly 
used in the theory of open systems. Open systems are physical systems, in classical 
or in quantum mechanics, which are not closed, that is, which are interacting with 
another system. In general the system we are interested in is "small" (for example, 
it has only a finite number of degrees of freedom), whereas the outside system is 
very large (often called the "environment" , it may be a heat bath typically). 

This is now a very active branch of research to study such systems coupled to an 
environment. In classical mechanics they are used to study conduction problems 
(Fourier's law for example, see [3], [5]) but more generally out of equilibrium dy- 
namics (see [14], [4]). In quantum mechanics, open systems appear fundamentally 
for the study of decoherence phenomena (see [8]), but also it is the basis of quan- 
tum communication (see [11]). Problems of dissipation, heat conduction, out of 
equilibrium dynamics in quantum mechanics (see [9], [10]) lead to very important 
problems which are mostly not understood at the time we write this article. 

The aim of this article is to make clear several ideas and connections between 
deterministic dynamics of closed systems, effective dynamics of open systems and 
Markov processes. 
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Surprisingly enough these ideas are made rather clear in the literature when 
dealing with the quantum systems, but not that much with classical ones! Indeed, 
it is common in quantum mechanics to consider a bipartite system on which one 
component is not accessible (it might be an environment which is too complicated 
to be described, or it might be Bob who is sharing the photons of a correlated 
pair with Alice, in Quantum Information Theory). It is well-known that, tracing 
out over one component of the system, the unitary Schrodinger dynamics becomes 
described by completely positive maps, in discrete time, or completely positive 
semigroups, in continuous time. 

In [2], for example, the authors show how every discrete time semigroup of 
completely positive maps can be described by a realistic physical system, called 
"Repeated Quantum Interactions" . They show that in the continuous time limit 
these Hamiltonian dynamics spontaneously converge to a dynamics described by 
a quantum Langevin equation. 

In this article we establish many similar results in the context of classical dy- 
namical systems and Markov chains. The article is structured as follows. In 
Section 2, we show that Markov chains appear from any dynamical system on a 
product space, when averaging out one of the two components. This way, Markov 
chains are interpreted as what remains on one system when it interacts with some 
environment but we do not have access to that environment. The randomness 
appears directly for the determinism, solely by the fact that we have lost some 
information. We show that any Markov chain can be obtained this way. We also 
show two results which characterize what properties are lost when going from a 
deterministic dynamical system to a Markov chain: typically the loss of algebra 
morphism property and the loss of reversibility. 

In Section 3 we explore the context of classical Markov process in the continuous 
time setup. We actually concentrate on stochastic differential equations. Despite 
of their "random character" , we show that stochastic differential equations are ac- 
tually deterministic dynamical systems. They correspond to a natural dynamical 
system which is used to dilate some Markov processes into a deterministic dy- 
namics. The role of the environment is played by the canonical probability space 
(here the Wiener space), the action of the environment is the noise term in the 
stochastic differential equation. 

2. Markov Chains and Dynamical Systems 

2.1. Basic Definitions. Let us recall some basic definitions concerning dynam- 
ical systems and Markov chains. 

Let (E, £) be a measurable space. Let T be a measurable function from E to 
E. We then say that T is a dynamical system on E. Such a mapping T induces a 
natural mapping T on C°°(E) defined by 

Tf(x) = f(fx) . 

Note that this mapping clearly satisfies the following properties (proof left to the 
reader). 
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Proposition 2.1. 

i) T is a *-homomorphism of the *-algebra C°°{E) 

n) T(i E ) = i E , 

Hi) \\T\\ = 1. 



What is called dynamical system is actually the associated discrete-time semi- 
group (T")„ 6 n, when acting on points, or (T")„gN, when acting on functions. 

When the mapping T is invertible, then so is the associated operator T. The 
semigroups (T")„ 6 n and (T")„ 6 n can then be easily extended into one-parameter 
groups (f") n£Z and (T")„ eZ , respectively. 

Let us now recall basic definitions concerning Markov chains. Let (E, £) be a 
measurable space. A mapping v from E x £ to [0, 1] is a Markov kernel if 

i) x i V v(x, A) is a measurable function, for all A e £ , 

ii) A i-> ^(x, A) is a probability measure, for all x G E. 

When E is a finite set, then v is determined by the quantities 



which form a stochastic matrix, i.e. a square matrix with positive entries and sum 
of each row being equal to 1. 

In any case, such a Markov kernel v acts on C°°(E) as follows: 



Je 

for some Markov kernel is, is called a Markov operator. 

In a dual way, a Markov kernel ^ acts on probability measures on (E, £). Indeed, 
if P is a probability measure on (E, £) then so is the measure f ov defined by 



Je 

Finally, Markov kernels can be composed. If v\ and V2 are two Markov kernels 
on (E, £) then so is 



This kernel represents the Markov kernel resulting from making a first step fol- 
lowing v\ and then another step following vi- 

A Markov chain with state space (E,£) is a discrete-time stochastic process 
(X n ) ne pi defined on a probability space (ft, J 7 , P) such that each X n : £1 — > E is 
measurable and 




A linear operator T on C°°(E, £) which is of the form 






E [f(X n+1 ) | X , Xi, . . . , X n ] = E [/ (A„ +1 ) | X n ] 
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for all bounded function / : E — >• K and all n G N. In particular, if F n denotes 
the cr-algebra generated by Xq, X\, . . . , X„, then the above implies 

E[f(X n+1 )\F n ]=L n f(X n ) 

for some function L n f. The Markov chain is homogeneous if furthermore L„ does 
not depend on n. We shall be interested only in this case and we denote by L this 
unique value of L n : 

E[f(X n+1 )\T n ]=Lf(X n ). (2.1) 

Applying successive conditional expectations, one gets 

E {f(X n ) | Jo] = L n f(X ) . 

If v(x, dy) denotes the conditional law of X n+ i knowing X n = x, which coincides 
with the conditional law of X\ knowing A = x, then v is a Markov kernel and 
one can easily see that 

Lf{x)= / f(y) v{x, dy) = v o f(x) . 
Je 

Hence L is the Markov operator associated to v. 

With our probabilistic interpretation we get easily that v o f(x) is the expecta- 
tion of f{X\) when X = x almost surely. The measure P o v is the distribution 
of X\ if the distribution of X is P. 

We end up this section with the following last definition. A Markov kernel v is 
said to be deterministic if for all x e E the measure u(x, • ) is a Dirac mass. This 
is to say that there exists a measurable mapping T : E — > E such that 

v(x,dy) = S f{x) (dy) . 

In other words, the Markov chain associated to v is not random at all, it maps 
with probability 1, each point x to T{x): it is a dynamical system. 

2.2. Reduction of Dynamical Systems. Now consider two measurable spaces 
(E,£) and (F,T), together with a dynamical system T on E x F, equipped with 
the product er-held. As above, consider the lifted mapping T acting on C°°{E x F). 

For any bounded measurable function / on E, we consider the bounded (mea- 
surable) function / ® 1 on E x F defined by 

(f®l)(x,y)=f(x), 

for all x G E, y G F. 

Assume that (F 7 F) is equipped with a probability measure fi. We shall be 
interested in the mapping L of C°°{E) defined by 

Lf(x)= f T{f®l){x,y)d f i{y)= f (f ® l)f(x,y) d»(y) . (2.2) 
Jf J f 

In other words, we have a deterministic dynamical system on a product space. 

We place ourselves from one component point of view only (we have access to E 

only). Starting from a point x G E and a function / on E we want to see how 

they evolve according to T, but seen from the E point of view. The function / on 

E is naturally lifted into a function / ® 11 on E x F, that is, it still acts on E only, 

but it is now part of a "larger world" . We make / ® 11 evolve according to the 
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deterministic dynamical system T . Finally, in order to come back to E we project 
the result onto E, by taking the average on F according to a fixed measure /x on F. 
This is to say that, from the set E, what we see of the action of the "environment" 
F is just an average with respect to some measure fi. 

Theorem 2.2. The mapping L is a Markov operator on E. 

Proof. As T is a mapping from E x F to E x F, there exist two measurable 
mappings: 

X : E x F — > E and Y : E x F — > F , 

such that 

T(x,y) = (X(x,y),Y(x,y)) 

for all (a;, y) £ E x F. 

Let us compute the quantity Lf(x), with these notations. We have 

Lf(x) = J T(f®l){x,y)drty) 

= [ f(X(x,y))d^y). 
Jf 

Denote by v(x, dz) the probability measure on E, which is the image of \x by the 
function X(x, • ) (which goes from F to E, for each fixed a;). By a standard result 
from Measure Theory, the Transfer Theorem, we get 

Lf(x)= f f{z)v{x,dz). 

J E 

Hence L acts on C°°{E) as the Markov transition kernel v(x,dz). □ 

Note the following important fact: the mapping Y played no role at all in the 
proof above. 

Note that the Markov kernel v associated to T restricted to E is given by 

v(x, A)=n {{y £ F; X(x, y) £ A}) . (2.3) 

In particular, when E is finite (or even countable), the transition kernel v is 
associated to a Markovian matrix P whose coefficients are given by 

P(i,j) = v(i,{j}) = V({k;X(i,k)=j}). 

What we have obtained here is important and deserves more explanation. Math- 
ematically, we have obtained a commuting diagram: 

T 

C°°{ExF) — ► C°°(ExF) 

m 

C°°(E) — > C°°(E). 
L 
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In more physical language, what we have obtained here can be interpreted in two 
different ways. If we think of the dynamical system T first, we have emphasized 
the fact that losing the information of a deterministic dynamics on one of the 
components creates a random behavior on the other component. The randomness 
here appears only as a lack of knowledge of deterministic behavior on a larger 
world. A part of the universe interacting with our system E is inaccessible to us 
(or at least we see a very small part of it: an average) which results in random 
behavior on E. 

In the converse direction, that is, seen from the Markov kernel point of view, 
what we have obtained is a dilation of a Markov transition kernel into a dynamical 
system. Consider the kernel L on the state space E. It does not represent the 
dynamics of a closed system, it is not a dynamical system. In order to see L as 
coming from a true dynamical system, we have enlarged the state space E with 
an additional state space F, which represents the environment. The dynamical 
system T represents the true dynamics of the closed system "_E+environment" . 
Equation (2.2) says exactly that the effective pseudo-dynamics L that we have 
observed on E is simply due to the fact that we are looking only at a subpart of 
a true dynamical system and an average of the F part of the dynamics. 

These observations would be even more interesting if one could prove the con- 
verse: every Markov transition kernel can be obtained this way. This is what we 
prove now, with only a very small restriction on E. 

Recall that a Lusin space is a measurable space which is homeomorphic (as a 
measurable space) to a Borel subset of a compact metrisable space. This condition 
is satisfied for example by all the spaces R". 

Theorem 2.3. Let (E,£) be a Lusin space and v a Markov kernel on E. Then 
there exists a measurable space (F, F), a probability measure ji on (F,F) and a 
dynamical system T on E x F such that the Markov kernel L associated to the 
restriction of T to E is equal to v. 

Proof. Let u(x,dz) be a Markov kernel on (E,£). Let F be the set of functions 
from E to E. For every finite subset a = {xx , . . . , x n } C E and every A\ , . . . , A n G 
£ consider the set 

F(xi, . . .,x n ; Ax, . . .,A n ) = {y e F ; y(xi) e Ax, . . . ,y(x n ) e A n } . 

By the Kolmogorov Consistency Theorem (which applies for E is is a Lusin space!) 
there exists a unique probability measure ^onF such that 

n 

fj, {F{xx,. ■ .,x n ;Ai,. . .,A n )) = \\_v{x t , A t ) . 

Indeed, it is easy to check that the above formula defines a consistent family of 
probability measures on the finitely-based cylinders of F, then apply Kolmogorov's 
Theorem. 

Now define the dynamical system 

f : E x F — > E x F 
(x,y) i — > (y(x),y). 
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With the same notations as in the proof of Theorem 2.2, we have X(x, y) = y(x) 
in this particular case and hence 

M ({y G F ■ X(x, y) G A}) = G F ; y{x) G A}) = v{x, A) . 

This proves our claim by (2.3). □ 

Note that in this dilation of L, the dynamical system T has no reason to be 
invertible in general. It is worth noticing that one can construct a dilation where 
T is invertible. 

Proposition 2.4. Every Markov kernel v, on a Lusin space E, admits a dilation 
T which is an invertible dynamical system. 

Proof. Consider the construction and notations of Theorem 2.3. Consider the 
space F' = E x F. Let x be a fixed element of E and define the mapping T' on 
E x F' by 

'T'(x, (x ,y)) = (y(x), (x, y)), 
T'(x, {y(x),y)) = (x , {x,y)), 

T'(x, (z, y)) = (z, (x, y)), if z ± x and z ^ y(x) . 

It is easy to check that T' is a bijection of E x F'. Now extend the measure /j, on 
F to the measure d Xo <g> /j, on F'. Then the dynamical system T" is invertible and 
dilates the same Markov kernel as T. □ 

2.3. Iterating the Dynamical System. We have shown that every dynamical 
system on a product set gives rise to a Markov kernel when restricted to one of the 
sets. We have seen that every Markov kernel can be obtained this way. But one 
has to notice that our construction allows the dynamical system T to dilate the 
Markov kernel L as a single mapping only. That is, iterations of the dynamical 
system T n do not in general dilate the semigroup L n associated to the Markov 
process. Let us check this with a simple counter-example. 

Put E — F = {1,2}. On F define the probability measure = 1/4 and 
(i(2) = 3/4. Define the dynamical system T on E x F which is the "anticlockwise 
rotation" : 

T(l,l) = (2,l), f(2,l) = (2,2), f(2,2) = (l,2), T(l, 2) = (1, 1) . 
With the same notations as in previous section, we have 

X(l,l) = 2, X(2,l) = 2, X(2,2) = l, X(l,2) = l. 
Hence, we get 

M (X(1, ■) = !) = \, u(X(l,-) = 2) = ±, 

MWV) = 1) = J, h(X{2,-) = 2) = \. 
Hence the Markovian matrix associated to the restriction of T to E is 
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In particular 

L 2 = L . 

Let us compute T 2 . We get 

T 2 (l,l) = (2,2), f 2 (2,l) = (l,2), T 2 (2,2) = (l,l), T 2 (l,2) = (2,l). 
Hence the associated X-mapping, which we shall denote by X 2 , is given by 

X 2 (l,l) = 2, X 2 (2,l) = l, X 2 (2,2) = l, X 2 (l,2) = 2. 
This gives the Markovian matrix 

-CD 

which is clearly not equal to L 2 . 

It would be very interesting if one could find a dilation of the Markov kernel L 
by a dynamical system T such that any power T n would also dilate L n . We would 
have realized the whole Markov chain as the restriction of iterations of a single 
dynamical system on a larger space. 

This can be performed in the following way (note that this is not the only way, 
nor the more economical). Let L be a Markov operator on a Lusin space E with 
kernel v and let T be a dynamical system on E x F which dilates L. Consider 
the set F — F N equipped with the usual cylinder a-field F m and the product 
measure /x = £i® N . The elements of F are sequences (y„) neN * in F. Put 

S : E x F — > E x F 

(x,y) .— > (X(x, yi ),Q(y)) 

where X is as in the the proof of Theorem 2.2 and 6 is the usual shift on F: 
Q(y) = (y„+i) neN *. 

Then S can be lifted into a morphism S of C°°{E x F), as previously. Fur- 
thermore, any function / in C°°(E) can be lifted into / ® 1 on C°°(E x F), with 
(/® !)(*,») = /(*)■ 

Theorem 2.5. For all nef, all x e E and all f G C°°(E) we have 

J^S n (f®H)(x,y)dn(y) = (L n f)(x). 

Proof. Recall that we noticed in the proof of Theorem 2.2, that the mapping Y 
associated to T played no role in the proof of this theorem, only the mapping X 
was of importance. In particular this implies that Theorem 2.5 is true for n = 1, 
for the dynamical systems T and S share the same X-mapping. 
By induction, let us assume that the relation 

j^S k (f ®K)(x,y)dii(y) = {L k f){x) 
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holds true for all / E £°°(E), all x £ E and all k < n. Set Fp to be the set of 
sequences (y n ) n>2 with values in F and j2[ 2 the restriction of ji to i*j 2 . We have 

^S n+l (f®1L)(x,y)di2(y) = 

= J^S n (j®t) (X(x, yi ),Q(y)) dfty) 

= [ [ S n (f(Sl)(X(x,y 1 ),y)dJl [2 (y)d^y 1 ). 

J F JF [2 

Put x = X(x,yi), the above is equal to 

/ f S n {f®t)(x,y)dfl [2 (y)d^y 1 ) 

JF JF [2 

L n (f)(x) dfi(yi) (by induction hypothesis) 

= J L n (f)(X(x, yi ))dn( yi ) 
= L n +\f)(x). 

□ 

With this theorem and with Theorem 2.3, we see that every Markov chain on E 
can realized as the restriction on E of the iterations of a deterministic dynamical 
system T acting on a larger set. 

The physical interpretation of the construction above is very interesting. It 
represents a scheme of "repeated interactions" . That is, we know that the result 
of the deterministic dynamics associated to T on E x F gives rises to the Markov 
operator L on E. The idea of the construction above is that the environment is 
now made of a chain of copies of F, each of which is going to interact, one after the 
other, with E. After, the first interaction between E and the first copy of F has 
happened, following the dynamical system T, the first copy of F stops interacting 
with E and is replaced by the second copy of F. This copy now interacts with 
E following T. And so on, we repeat these interactions. The space E keeps the 
memory of the different interactions, while each copy of F arrives independently 
in front of E and induces one more step of evolution following T. 

As a result of this procedure, successive evolutions restricted to E correspond 
to iterations of the Markov operator L. This gives rise to behavior as claimed: an 
entire path of the homogeneous Markov chain with generator L. 

2.4. Defect of Determinism and Loss of Invertibility. We end up this sec- 
tion with some algebraic characterizations of determinism for Markov chains. The 
point is to characterize what exactly is lost when going from the deterministic 
dynamics T on E x F to the Markov operator L on E. 

Theorem 2.6. Let (E,£) be a Lusin space. Let (X n ) be a Markov chain with 
state space (E,£) and with transition kernel v. Let L be the Markov operator on 
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C°°(E) associated to v: 

Lf(x)= [ f{y)v{x,dy). 

J E 

Then the Markov chain (X n ) is deterministic if and only if L is a *-homomorphism 
of the algebra C X {E). 

Proof. If the Markov chain is deterministic, then L is associated to a dynamical 
system and hence it is a *-homomorphism (Proposition 2.1). 

Conversely, suppose that L is a *-homomorphism. We shall first consider the 
case where (E, £) is a Borel subset of a compact metric space. 
Take any A G £ 1 any x e E and recall that we always have 

u(x,A) = L(l A )(x). 

The homomorphism property gives 

L(l A )(x) = L(l 2 A )(x) = L(lU) 2 (x) - V (x, A) 2 . 

Hence v{x, A) satisfies v{x,A) 2 = v{x,A). This means that v{x,A) is equal to 
or 1, for all x G E and all A G £ . 

Consider a covering of E with a countable family of balls (Bi) ie ^ 7 each of 
which with diameter smaller than 2~™ (this is always possible as E is separable). 
From this covering one can easily extract a partition (S'i)ieN °f E by measurable 
sets, each of which with diameter smaller than 2~". We shall denote by <S" this 
partition. 

Let x G E be fixed. As we have X^Be<S" v k x i E) — 1 we must have v(x, E) = 1 
for one and only one E G S n . Let us denote by E^ n \x) this unique set. Clearly, 
the sequence (E^(x)) neN is decreasing (for otherwise there will be more than 
one set E G S n such that v(x,E) = 1). Let A = n n E^(x). The set A satisfies 
v(x, A) — 1, hence A is non-empty. But also, the diameter of A has to be 0, for it 
is smaller than 2~" for all n. As a consequence A has to be a singleton {y(x)}, for 
some y(x) G E. Hence we have proved that for each x G E there exists a y(x) G E 
such that n{x, {y(x)}) = 1. This proves the deterministic character of our chain. 

The case where E is only homeomorphic to a Borel subset E' of a compact met- 
ric space is obtained by using the homeomorphism to transfer suitable partitions 
S n of E' to E. □ 

The result above is quite amazing. It gives such a clear and neat characterization 
of the difference between a true Markov operator and a deterministic one! One can 
even think of several applications of this characterization, for example one may be 
able to measure the "level of randomness" of some Markov operator by evaluating 
for example 

sup{||T(/ 2 ) - T{}) 2 \\ ■ f G C°°(E,e), H/ll = 1} . 

I do not know if such things have already been studied or not. It is not my purpose 
here to develop this idea, I just mention it. 
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Another strong result on determinism of Markov chains is the way it is related 
to non-invertibility. 

Theorem 2.7. Let (E, £) be a Lusin space. Let L be a Markov operator on C°°(E) 
associated to a Markov chain (X n ). If L is invertible in the category of Markov 
operators then (X n ) is deterministic. 

Proof. Recall that a Markov operator L maps positive functions to positive func- 
tions. Hence, in the same way as one proves Cauchy-Schwarz inequality, we always 
have 

Hf) = L(J) 

and 

L(\f\ 2 ) > Hf)L(f) 
(hint: write the positivity of T((f + Xg)(f + Xg)) for all A G C). 
Let M be a Markov operator such that ML = LM — I. We have 

|/| 2 = // = M o L(ff) > M(L(f)L(f)) > M o L(f) M o L(f) = ff = |/| 2 . 
Hence we have equalities everywhere above. In particular 

MoL(ff) = M(L(f)L(f)). 
Applying L to this equality, gives 

L(ff) = L(f)L(f) , 

for all / e C°°{E). 

By polarization it is easy to prove now that L is a homomorphism. By Theorem 
2.6 it is the Markov operator associated to a deterministic chain. □ 

The result above is more intuitive than the one of Theorem 2.6, from the point 
of view of open systems. If the dynamical system T on the large space E x F is 
invertible, this invertibility is always lost when projecting on E. The fact we do 
not have access to one component of the coupled system makes that we lose all 
chance of invertibility. 

3. Continuous Time 

We now leave the discrete-time setup to concentrate on continuous-time dynam- 
ical systems. We aim to show that stochastic differential equations are actually 
a particular kind of continuous-time dynamical systems. In particular they are 
"deterministic". The type of dynamical system we shall obtain this way is a 
continuous-time version of the construction of Theorem 2.5. 

3.1. Preliminaries. Let us consider the d-dimcnsional Brownian motion W on 
its canonical space (f2, J 7 , P). This is to say that f2 = Cb(lR + ;K d ) is the space 
of continuous functions on M + with values in R d and which vanish at 0, equiped 
with the topology of uniform convergence on compact sets, the cr-field T is Borcl 
CT-field of fi and the measure P is the Wiener measure, that is, the law of a d- 
dimensional Brownian motion on ft. The canonical Brownian motion (Wt) is 
defined by W t (uj) = u)(t), for all w € O and all t G K+. This is to say, coordinate- 
wise: Wl (bS) — Wi(t), for i = 1, . . . , d. 
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We define for all s G M + the shift 8 S as a function from il to fi by 

O s (w)(t) =uj(t + s) -lj(s). 

We define the shift operator Q s as follows. If X is any random variable on Q we 
denote by O s (J*f ) the random variable X o Q s , whatever is the state space of X. 
In particular we have & s (W t ) = W t+S — W s . 

As the process Y t = Wt+ S — W s , t € M. + , is again a d-dimcnsional Brownian mo- 
tion, this implies that the mapping S preserves the measure P. As a consequence 
6 S is an isometry of L 2 ((fi, F, P); R d ). 

Lemma 3.1. If H is a predictable process in Mr, then, for all fixed s € R + , the 
process K t = O s (H t - s ), t > s is also predictable. 

Proof. The process K as a mapping from £1 x [s, +oo[ to R d is the composition of 
H with the mapping <j>(oj, t) = (8 s (uj),t — s) from ft x [s, +oo[ to ft x K+. We just 
need to check that <j> is measurable for the predictable tr-algebra P. 

Consider a basic predictable set ix]«, w], with u < v and A G J- u , then 

(f)- 1 {Ax] Ul v}) = e-\A) x]u + s, v + s] . 

We just need to check that 6*7 1 (J r u ) C J- u +s- The cr-algcbra T u is generated by 
events of the form (W{ £ [a, &]), for t < u and i = 1, . . . , d. The set 9j 1 (W^ G [a, 6]) 
is equal to (W t ' +S — W l s e [a, 6]), hence it belongs to J- u+s . 
One needs also to note that 

<F\A x {0}) = e~\A) x {s} eJ s x {s} 

for all A E J-q. We have proved the predictable character of K. □ 



In the following, the norm ||-|| 2 is the L 2 ((0, J", P); M d )-norm. For a Revalued 
predictable process If we put 



/ H s -dW s =J2 
Jo i=1 Jo 



HI dWl . 



Lemma 3.2. Let H be a predictable process in R d such that J Q s ||i? M || 2 du < oo. 
Then we have 

e s h u ■ dw}j - J + e s (H u _ s ) ■ dw u . (3.i) 

Proof. If H is an elementary predictable process then the identity (3.1) is obvious 
from the fact that Q S (FG) — @ S (F)Q S (G) for any scalar-valued F and G. A 
general stochastic integral J* H s ■ dW s is obtained as a limit in the norm 







/ H s ■ dW s 




Jo 


2 JO 



\H s \\ 2 ds. 



of stochastic integrals of elementary predictable processes. As @ s is an isometry, 
it is clear that Equation (3.1) holds true for any stochastic integral. □ 
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Here comes now the main result of this section. Before hands recall the following 
result on stochastic differential equations (cf [13], Chapter V). Let / be a locally 
bounded Lipschitz function from R™ to R™ and g a locally bounded Lipschitz 
function from R™ to M„ X( j(R). Consider the stochastic differential equation 



Xf = x + 



f f{XZ)du+ f g{XZ)-dW Ul 
Jo Jo 



which is a shorthand for 



o 



i=i Jo 



g(X:y r dWi, 



for all i = 1, . . . ,n. Then this equation admits a solution X x and this solution 
is unique, in the sense that any other process on (Q, T , P) satisfying the same 
equation is almost surely identical to X x . 

Theorem 3.3. Let W be a d- dimensional Brownian motion on its canonical space 
(f2, T, P). Let f be a locally bounded Lipschitz function from R™ to R" and g a 
locally bounded Lipschitz function from R" to M„ X( j(R). Denote by X x the unique 
stochastic process (in W 1 ) which is a solution of the stochastic differential equation 

X? = x+ f f{X x u ) du + f g{X x u ) ■ dW u . 
Jo Jo 

Then, for all s G R+, for almost all uj £ SI, we have, for all t G R+ 

Xr J "\9 s {uj))=X x +t {uj). 

Remark: Let us be clear about the sentence "for all s G R + , for almost all w£fl, 
we have, for all t G R + " above. It means that for all s G R + , there exists a null-set 
7V S C il such that for all w G il \ Af s we have, for all t G R + ... 



Proof. Let s be fixed. Define, for all u> G 

'ATM 



X^\9 s (uj)) 



if u < s , 
if u > s . 



Then Yf +t satisfies 



= ATH + 

= x + 
+ 



Jo 



) du 



Jo 



■dW u 



f(K)du 



g{K) ■ dw u 



(«)+ 



J* f{Xp){O a ( U )) du + jT' +t Q s (g(xZ' a )) ■ dW u 



by Lemma 3.2. 
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Now, coming back to the definition of Y we get 



Y s * +t (u)=x + 



f f(Y:)(u) du+\f g(Y:) ■ dW u ] M + f f(Y^,){u) du+ 
Jo IJo J Jo 



s+t 



g(x:) ■ dw u 



= x + f f(Y:)du+ f g(Y*)-dW u + f f(Y:)du+ 

JO Jo Js 

f + g(YZ) ■ dW u 

J s 



+ 



= X + 



f S+t f(YZ)du+ f S+t g(Y:) • dW u 
Jo Jo 



This shows that Y x is solution of the same stochastic differential equation as X x . 
We conclude easily by uniqueness of the solution. □ 

3.2. Stochastic Differential Equations and Dynamical Systems. We are 

now ready to establish a parallel between stochastic differential equations and 
dynamical systems. Recall how we defined discrete time dynamical systems T in 
Section 2 and their associated semigroups (T n ). In continuous time the definition 
extends in the following way. 

A continuous-time dynamical system on a measurable space (E, £) is a one- 
parameter family of measurable functions (T , t ) t6R + on E such that T s o T t = T s+t 
for all s,t. That is, T is a semigroup of functions on E. 

Each of the mappings T t can be lifted into an operator on £°° (E) , denoted by 
T t and defined by 

T t f(x) = f(T t x). 
The following result is now a direct application of Theorem 3.3. 

Corollary 3.4. Let W be a d-dimensional Brownian motion on its canonical space 
(O, J 7 , P) . Let f be a locally bounded Lipschitz function from K" to M™ and let g be 
a locally bounded Lipschitz function from 1" to M„ X d(K). Consider the stochastic 
differential equation (on R"J 

x; = x + f f(xz) du + f g (xz) ■ dw u . 

Jo Jo 

Then the mappings T t on K" x f2 defined by 

T t (x,u;) = (X?(u;),6t(u;)) 

define a continuous time dynamical system on K" x f2, in the sense that there 
exists a null set N <Z £l such that for all uj G Sl\ M, for all x e R" and for all 
s,t G R + we have 



T t oT s (x,uj) = T s oT t (x,u>) = T s+t (x,u>) . 



MARKOV CHAINS AND DYNAMICAL SYSTEMS 



15 



Proof. The null set TV appearing in Theorem 3.3 also depends on the initial point 
x G M™. Let us denote by TV XjS this set, instead. Let Af x , s ,t be the null set 
j\T XyS U M x ,t- Finally put 

M = U U N x ,s,t ■ 

x€Q n s,t£Q+ 

Then TV is a null set and for all cj G Cl \ TV the relations 

T t oT s (x,uj) =T s or,M = T s+t (x,u) 

hold true for all x G Q™ and all s, t G Q + , by Theorem 3.3. 

The solution Xf (w) is continuous in t, except for a null set TV' of cj's. Hence, 
by continuity, the relations above remain true for all s, t G M + , if w G \ (TV U TV) . 

In the same way, as the solution Xf depends continuously in x, we conclude 
easily. □ 

This is to say that, apart from this minor restriction to the complementary of a 
null set in Q, a stochastic differential equation is nothing more than a deterministic 
dynamical system on a product set R" x ft, that is, it is a semigroup of point 
transformations of this product set. 

We now have a result analogous to the one of Theorem 2.5 when this dynamical 
system is restricted to the JR™-component. But before establishing this result, we 
need few technical lemmas. In the following f2 t ] denotes the space of continuous 
functions from [0,t] to M. d . For all oj G fi we denote by w t ] the restriction of ui to 
[0,f]. Finally P t ] denotes the restriction of the measure P to (£l t j , T t ). 

Lemma 3.5. The image of the measure F under the mapping 

— > fl t i x!1 

is the measure P t ] ® P. 

Proof. Recall that oj(s) = W s (to) and 9 t (u>)(s) = W t + s (oj) - W t (oj). If A is a finite 
cylinder of Sl t ] and B a finite cylinder of SI, then the set 

{uj G CI; (uj t] ,O t (uj)) EAxB} 

is of the form 

{^Gtt; W tl (oj) G Ai,. . . , W tn (to) G A n , (W Sl - W t )(co) G Bi, . . . 

...,{W 8h -W t )(Lj)€B k } 

for some t\,...,t n < t and some si,...,Sk > t. By the independence of the 
Brownian motion increments, the probability of the above event is equal to 

P t] ({w G fl t] ; W tl (w) G A u ...,Wt n {uj) G A n })x 

x P({^ G ; (W S1 - W t )(w) 6 5!,., (M/ Sfc - W t )(w) G B k }) . 

This is to say, 

P({wgO; (w(],D ( (w))e4xB}) = 

= P t] ({w t] G O t] ; w t] G ^})P({c^ efi;9 t He B}) . 
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This is exactly the claim of the lemma for the cylinder sets. As the measures P and 
P t ] (£>P are determined by their values on the cylinder sets, we conclude easily. □ 

Lemma 3.6. Let g be a bounded measurable function on t ] x ft. Then we have 

f f g(co,oj')dP t] (co)dP(uj')= [ g(co t] ,e t (uj))dP(co). 
Jn Jn t] Jn 

Proof. This is just the Transfer Theorem for the mapping of Lemma 3.5. □ 

Theorem 3.7. Let (f2, J 7 , P) be the canonical space of a d- dimensional Brownian 
motion W . Let f be a locally bounded Lipschitz function from W 1 to K™ and let g be 
a locally bounded Lipschitz function from W 1 to M„ X d(R). Consider the stochastic 
differential equation (onW 1 ) 

X? = x+ ( f{XZ)du+ f g{XZ)-dW u 
Jo Jo 

and the associated dynamical system 

T t (x,u) = (X?(Lj),6 t (uj)) . 
For any bounded function h on M™ consider the mapping 

P t h(x)=E[T t (h®l)(x, ■)]= [ h(X%{w))dF{uj). 

Jn 

Then (Pt)tt=m+ is a Markov semigroup on R" with generator 

n „ , n d „ 2 

i— 1 a—1 J 

Proof. The fact that each P t is a Markov operator is a consequence of Theorem 
2.2. Let us check that they form a semigroup. 

First of all note that, since X x is a predictable process, the quantity Xf{u) 
depends only on uj t ] and not on the whole of oj. We shall denote by (w t ]) the 
associated function of cj t ] . 

By definition of P t we have 

P t (P s h){x)=E[T t (P s h®-SL)(x, •)] 

= / P s h(X?(w))dF(uj) 
Jn 

= jj h(xf [w \j)) dF(uj') dP(iu) 

= f [ hix^^iu: 1 )) dF{oj')dF t] {uj) 
Jn t] Jn v ' 

= J h (xf f(wtl) (0 t (u;))) cflP(w) (by Lemma 3.6) 

= / h(X x s+t {w)) d¥(uj) (by Theorem 3.3) 
Jn 

= P s+t h{x) . 
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We have proved the semigroup property. 

The rest of the proof comes from the usual theory of Markov semigroups and 
their associated generators (see for example [12], Chapter VII). □ 

We have proved the continuous time analog of Theorem 2.5. Every Markov 
semigroup, with a generator of the form above, can be dilated on a larger set (a 
product set) into a deterministic dynamical system. What is maybe more surpris- 
ing is that the deterministic dynamical system in question is a stochastic differen- 
tial equation. Theorem 3.7 and Corollary 3.4 show that a stochastic differential 
equation can actually be seen as a particular deterministic dynamical system. 

Theorem 3.7 above again gives an open system point of view on Markov pro- 
cesses: Markov processes are obtained by the restriction of certain types of dy- 
namical systems on a product space, when one is averaging over one inaccessible 
component. The role of the environment is now played by the Wiener space and 
the role of the global dynamics on the product space is played by the stochastic 
differential equation. 

In this section we have developed the Brownian case only. But it is clear that all 
this discussion extends exactly in the same way to the case of the Poisson process. 
Indeed, the arguments developed above are mostly only based on the independent 
increment property. 

We have said that stochastic differential equations are particular dynamical sys- 
tems which are continuous analogues of those of Section 2.3: repeated interactions. 
In the article [7] , the convergence of discrete-time repeated interactions models to 
stochastic differential equations is proved. 

Comment. We do not pretend that all the results presented in this article are 
new. Let us be clear about that. The fact that restrictions of dynamical systems 
can give rise to Markov chains is rather well-known among specialists of dynamical 
systems. The results of Subsection 2.4 are adaptations to the classical context of 
similar results on completely positive maps for quantum systems. The fact that 
stochastic differential equations give rise to deterministic dynamical systems is also 
not new and can be found for example in [6] (see also [1] for more general noises). 

The originality of our article lies more in its survey character, in the way we 
put all these results together, in the connection we make with repeated interaction 
systems and in the physical point of view we adopt. 

Acknowledgment. The author is very grateful to the referee of this article for 
his very carefull reading, his remarks and suggestions. 
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